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Abstract — In coding schemes for the wire-tap channel or the 
broadcast channels with confidential messages, it is well known 
that the sender needs to use a stochastic encoding to avoid the 
information about the transmitted confidential message to be 
leaked to an eavesdropper. In this paper, it is investigated that 
the trade-off between the rate of the random number to realize 
the stochastic encoding and the rates of the common, private, and 
confidential messages. For the direct theorem, the superposition 
coding scheme for the wire-tap channel recently proposed by Chia 
and El Gamal is employed, and its strong security is proved. The 
matching converse theorem is also established. Our result clarifies 
that a combination of the ordinary stochastic encoding and the 
channel prefixing by the channel simulation is suboptimal. 

Index Terms — Broadcast Channel, Confidential Messages, Ran- 
domness Constraint, Stochastic Encoder, Superposition Coding, 
Wire-tap Channel 



I. Introduction 

The wire-tap channel is one sender and two receivers 
broadcast channel model in which the sender, usually referred 
to as Alice, wants to transmit a confidential message to the 
legitimate receiver, usually referred to as Bob, in such a way 
that the other receiver, usually referred to as eavesdropper Eve, 
cannot get any information about the transmitted message. The 
wire-tap channel model was first introduced by Wyner in his 
seminal paper (TJ. Later, Csiszar and Korner investigated the 
model called broadcast channels with confidential messages 
(BCC) in which Alice also sends a common message that 
is supposed to be decoded by both Bob and Eve. These 
models were further investigated by many researchers from 
theoretical point of view (e.g.. see 121), and recently it has 
attracted considerable attention from practical point of view 
as a physical layer security. 

In coding schemes for the wire-tap channel or the BCC, it is 
well known that the sender needs to use a stochastic encoder 
to avoid the information about the transmitted confidential 
message to be leaked to Eve. The stochastic encoding is 
usually realized by preparing a dummy random number in 
addition to the intended messages and by encoding them 
to a transmitted signal by a deterministic encoder. Further- 
more, when the channel to Bob is not more capable than 
the channel to Eve, it is known that the sender needs to 
use the channel prefixing to achieve the capacity region (or 
the secrecy capacity) because the capacity formulas involve 
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such a channel from an auxiliary random variable to the 
random variable describing the input signal of the channel 
101 . In literatures, it is assumed that there exists a channel 
realizing the channel prefixing. But in practice the prefixing 
channel must be simulated from a random number by using 
a method such as the channel simulation [4|, which usually 
involves certain amount of simulation error depending on the 
amount of the random number. So far, there was no paper 
investigating how much random number is needed to achieve 
the capacity region. Since the random number is precious 
resource in practice, though it has been paid no attention in 
literatures, it is extremely important to investigate the amount 
of random number needed to achieve the capacity region. 
For this purpose, we formulate the problem of the BCC by 
randomness constrained stochastic encoder, and completely 
characterize the capacity region of this new problem. 

The present problem to consider the randomness constrained 
stochastic encoder is motivated by the authors' previous results 
in (5]. In that paper, the capacity region of the relay channel 
with confidential messages for the completely deterministic 
encoder was investigated, and the capacity region was char- 
acterized for the BCC as a corollary. In this paper, we are 
interested in the case such that the randomness is constrained 
but not zero. The result in J5] can be regarded as an extreme 
case of the present problem. On the other hand, the conven- 
tional BCC problem can be regarded as the other extreme case, 
in which the amount of randomness that can be used at the 
encoder is unbounded. 

Typically in the BCC, Alice sends the common message 
that is supposed to be decoded by both Bob and Eve, and 
the confidential message that is supposed to be decoded only 
by Bob. The level of secrecy of the confidential message 
is usually evaluated by the equivocation rate. In this paper, 
we consider slightly different problem formulation, which has 
been appeared in the literature (6), 0. In our problem setting, 
Alice sends three kinds of messages, the common message, the 
private message, and the confidential messages. The common 
message is supposed to be decoded by both Bob and Eve. The 
private message is supposed to be decoded by Bob, and we do 
not care whether Eve can decode the private message or not. 
The confidential message is supposed to be decoded by Bob, 
and it must be kept completely secret from Eve. Furthermore, 
for stochastic encoding, Alice is allowed to use limited amount 
of dummy randomness. Thus, we are interested in the trade-off 
between quadruple of rates, the rate of dummy randomness, 
the rates of common, private, and confidential messages. The 
coding system of our formulation is depicted in Fig. Q] 

The reason we do not use the equivocation rate formulation 
is as follows. In the conventional equivocation rate formula- 
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Fig. 1. The coding system investigated in this paper. Alice sends common 
message K n , private message L n , and confidential message Sn by using a 
deterministic function f n and a limited amount of dummy randomness A n . 
The common message is supposed to be decoded by both Bob and Eve. 
The private message is supposed to be decoded by Bob, and we do not care 
whether Eve can decode the private message or not. The confidential message 
is supposed to be decoded by Bob, and it must be kept completely secret from 
Eve. 



tion, if the rate of dummy randomness is not sufficient, a part 
of the confidential message is sacrificed to make the other part 
completely secret and the rate of the completely secret part 
corresponds to the equivocation rate. We think that the rates 
of sacrificed part and completely secret part become clearer 
by employing our formulation. 

As we have mentioned above, the typical coding scheme 
for the wire-tap channel or the BCC consists of the stochastic 
encoding and the channel prefixing. In |7j, Chia and El 
Gamal proposed an alternative coding scheme that utilizes 
the so-called superposition coding |8) instead of the channel 
prefixing. In the direct part of our main result, we employ 
this superposition scheme instead of the channel prefixing. 
We also clarifies that a straightforward combination of the 
ordinary stochastic coding and the channel prefixing by the 
channel simulation method is suboptimal. 

Although Chia and El Gamal showed that the superposition 
coding scheme can realize the so-called weak security criterion 
0, iflOl . it was not clear whether the superposition coding 
scheme can realize the so-called strong security criterion. One 
of technical contributions of this paper is to show that Chia 
and El Gamal's superposition coding scheme can realize the 
strong security criterion. This is done by using the technique 
proposed in ifTTI . and by considering the channel resolvability 
problem |12| with the superposition coding. Note that the 
relationship between the wire-tap channel coding and the 
channel resolvability was first pointed out by Csiszar IflOl . 
and is well recognized recently 0~3), lfl4l . fl31 . 1161 , The 
channel resolvability with the superposition coding was first 
investigated by the second author in ATI . In that paper, the 
channel resolvability problem with the superposition coding 
for the degraded broadcast channel was considered to show the 
converse theorem of the identification via degraded broadcast 
channels. In this paper, the channel resolvability problem with 
the superposition coding for a single channel is considered. 
Using the superposition coding for a single channel seems 
nonsense at first glance, it does have a meaning when applied 
to the wire-tap channel or the BCC. 

After the submission of the first manuscript of this paper, 
we noticed some related works investigating the importance 



of random number in the BCC or the wire-tap channel. In 
AJ3, Hayashi and Matsumoto considered the secure multiplex 
coding lfl9l in which the messages are not necessarily uniform 
nor independent and the entropy rate of the messages might 
be constrained. Although the secure multiplex coding can be 
regarded as a generalization of the BCC, the encoder can 
use unlimited amount of dummy randomness in addition to 
the messages in their problem formulation. Thus, our results 
cannot be derived from their results. In ll20l . Bloch and 
Kliewer considered the wire-tap channel in which the dummy 
randomness is constrained and not necessarily uniform. How- 
ever, they only considered the case such that the channel to 
Bob is more capable than that to Eve. In such a case, the 
channel prefixing is not needed, and their result corresponds 
to Corollary [12] in this paper when the dummy randomness is 
uniform. 

The rest of the paper is organized as follows. In Section 
iHl the problem formulation is explained and main results are 
presented. In Section [HI] the channel resolvability problem 
with the superposition coding is discussed. In Section [IV] the 
proof of the main theorem is presented. In Section[VJ the paper 
is concluded with discussions. Some technical arguments are 
presented in Appendices. 

II. Problem Formulation and Main Results 
A. Problem Formulation 

Let Py\x an d Pz\x be two channels with common input al- 
phabet X and output alphabets y and Z respectively. Through- 
out the paper, the alphabets are assumed to be finite though we 
do not use finiteness of the alphabet except cardinality bonds 
on auxiliary random variables. We also assume that the base 
of logarithm is e throughout the paper. 

Let JC n be the set of the common message, C n be the set 
of the private message, and S n be the set of the confidential 
message. The common message is supposed to be decoded 
by both Bob and Eve. The private message is supposed to 
be decoded by Bob, and we do not care whether Eve can 
decode the private message or not. The confidential message 
is supposed to be decoded by Bob, and it must be kept 
completely secret from Eve. 

Typically, Alice use a stochastic encoder to make the confi- 
dential message secret from Eve, and it is practically realized 
by using a uniform dummy randomness on the alphabet A n . 
When the size \A n \ of dummy randomness is infinite, any 
stochastic encoder from K n x C n x S n to X n can be simulated 
by a deterministic encoder /„ : K n x C n x S n x A n — > X n . 
But we are interested in the case with bounded size \A n \ in 
this paper. 

Bob's decoder is defined by function g n : y n — > JC n x C n x 
S n and the error probability is defined as 



Perr\fnt 9n) 
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where 1 [•] is the indicator function. Eve's decoder is defined by 
function <p n : Z n — > K n and the error probability P err (/ n , <fi n ) 
is defined in a similar manner as Eq. (fTJ. 
Let 
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Rz\x( Z \fn(k n ,^m s n, a n)), 



be the output distributions of the channel P^\x- m tn ^ s P a P er ' 
we consider the security criterion given by 

D(f n ) := D(P Sn z n \\P Sn xP in ) 

= E (-1^)11^) 

= i"(5„;l ra ), 

where -D(-||-) is the divergence, and /(•;•) is the mutual 
information (8). The coding system investigated in this paper 
is depicted in Fig. Q] 

In this paper, we are interested in the trade-off among the 
rate the dummy randomness, and the rates of the common, 
private, and confidential messages. 

Definition 1: The rate quadruple (Rd, Ro, Ri, R s ) is said to 
be achievable if there exists a sequence of Alice's deterministic 
encoder /„ : JC n x £„ x S n x A n — > X n , Bob's decoder 
x C n x S n , and Eve's decoder <f> n : Z n — > K, n 

such that 



lim Perr(fn,g n ) 
n—too 

lim Perr{fn,<t>n) 
n—^oo 

lim D(f n ) 



n— >oo 

l 



o. 

0. 
0. 



lim sup — log \A„ 



1 



lim inf — log | /C n 

n— ^oo fl 

lim -log|£„| 

n— >oo n 

lim inf — log | <S„ | > R s . 



< Rd, 

> Ro, 
= Ri, 



(2) 
(3) 
(4) 

(5) 

(6) 

(7) 

(8) 



Then the achievable region 1Z is defined as the set of all 
achievable rate quadruples. 



B. Statements of General Results 

The following is our main result in this paper. 

Theorem 2: Let 1Z* be a closed convex set consisting 
of those quadruples (Rd, Ro, Ri, R s ) for which there exist 
auxiliary random variables (U, V) such that U <H> V O X O 



(Y, Z) and 

R < min[I{U;Y),I(U;Z)}, (9) 

Ro + Ri+Rs < I(V;Y\U)+mm[I(U;Y),I(U;Z)}, 

(10) 

Rs < I(V;Y\U)-I(V;Z\U), (11) 

Ri+Rd > I{X;Z\U), (12) 

R d > I(X;Z\V). (13) 

Then we have 1Z = 1Z*. Moreover, it may be assumed that 
V = (U, V) and that the ranges of U and V' may be assumed 
to satisfy \U\ <\X\+3 and |V'| < \X\ + 1. 

Proof: See Section [TV] ■ 
The conditions on Rq and i?i + R s in Eqs. (0 and (TTOb re- 
semble the conditions in the broadcast channel with degraded 
message sets 1211 . The condition on R s in Eq. (fTTT i exists 
because there is a security requirement on the confidential 
message. These conditions are exactly the same as those in 
the conventional BCC (see Corollary [6j. The conditions on Ri 
and Rd in Eqs. (fT2l and ( fT3l additionally appear in Theorem 
|2] because there are randomness constraints in our problem 
setting. 

Remark 3: Conventionally, the security requirement defined 

by 



lim i £>(/«) = 

n^-oo fl 



(14) 



is usually employed instead of Eq. (|4j. Eq. ( TPfl ) is called weak 
security criterion and Eq. is called strong security criterion 
ifTOl . |9l. Let 1Z be the achievable region in which Eq. is 
replaced by Eq. (TPfl) . From the definitions of two regions, 
1Z C 7Z obviously holds. Actually, we are implicitly showing 
TZ C 1Z* in the converse proof of Theorem [2] Thus, 1Z = 1Z. 

Remark 4: As we will find in the achievability proof of the 
main theorem, the private message can be used as dummy 
randomness to protect the confidential message from Eve. 
Thus, if we define the achievable rate region TZ by replacing 
Eq. © with 



lim inf — log \C n \ > R\, 

n— too TL 



(15) 



region TZ is broader than region TZ. Indeed, TZ is a closed 
convex set consisting of those quadruples (Rd, Ro, Ri,R s ) for 
which there exist auxiliary random variables (U, V) satisfying 
the same conditions as Theorem [2] except Eq. (TT2TI 1 L 

Remark 5: Eq. ( TPT i means that there is a certain amount of 
dummy randomness that cannot be substituted by the private 
message. Note that the difference between the private message 
and the dummy randomness is whether Bob needs to decode 
it or not. 

Let 

TZoo = {(Ro, Ri, Rs) '■ ^Rd > s.t. (Rd, Ro, Ri,R s ) £ TZ} 

'it can be proved by just omitting the derivation of Eq. U 2\ in the converse 
proof of Theorem f2] 
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be the set of all achievable triplet (Rq, R%, R s ) by arbitrary 
stochastic encoder. By taking sufficiently large R d , we recover 
the following well known result ||3lFi 

Corollary 6: (Q) Region IZoo is a closed convex set con- 
sisting of those triplet (Rq, Ri, R s ) for which there exist 
auxiliary random variables (U, V) such that U <-> V <-> X <-> 
(Y, Z) and 



Ra 



< 



i[I(U;Y),I(U;Z)], 



R + R 1 + R s < I(V;Y\U)+min[I(U;Y),I(U;Z)}, 
R s < I(V;Y\U)-I(V;Z\U). 

Let 

T^det = {(Ra, Ri,R s ) '■ (0, Ro, Ri,R s ) € 1Z} 

be the set of all rate triplets that can be achieved by determin- 
istic encoder. This extreme case was solved in [5|, which can 
be also derived as a corollary of Theorem [fl 

Corollary 7: (J3)) Let lZ* det be a closed convex set con- 
sisting of those triplet (Ro, Ri, R s ) for which there exists an 
auxiliary random variable U such that U -H- X -B- (Y, Z) and 

Ro < min[I(U;Y),I(U;Z)}, 

Ro + Ri+Rs < I(X;Y\U)+mm[I(U;Y),I(U;Z)], 

R s < I(X;Y\U)-I(X;Z\U), 

Ri > I(X;Z\U). 



Then we have 7Z det = lZ* det . 

Proof: The inclusion lZ* det C lZ de t is obvious by taking 
V = X in Theorem [2] For the opposite inclusion, note that 
Eq. (IT3b and R d = imply 

I(X; Z\U) = I(V; Z\U) + I(X; Z\V) = I(V; Z\U). 

We also have I(V;Y\U) < I(X\Y\U) from the Markov 
condition of the auxiliary random variables. Thus, we have 



K det c {(R ,Ri,R s ) : (0,R ,Ri,R s ) G 1Z*} c 1Z 



det ' 



Let 

Rd(Ro, Rs) = inf {Rd ■ (Rd, Rq, 0, R s ) s 11} 

be the infimum rate of dummy randomness needed to achieve 
the rates (Rq,R s ). From Theorem |2 we can characterize not 
only the known extreme cases (Corollary [6] and Corollary [7]l 
but also this quantity. 

Corollary 8: R d (R ,R s ) is the optimal solution of the 
following optimization problem: 

minimize I(X;Z\U) 
subject to 

R < min[I(U;Y),I(U;Z)}, 

Ro + Rs < I(V]Y\U)+mm[I(U;Y)J(U;Z)], 

Rs < I(V;Y\U)-I(V;Z\U), 

2 See also [6] Theorem 17.13] for the result that does not employ the rate- 
equivocation formulation. 

3 In (5), slightly deferent problem formulation is employed and the achiev- 
able region seems slightly different from Corollary [7] But they are essentially 
the same. 



where (U, V) satisfy U o V o- X O (Y, Z). 
Let 

K ds = {(R d ,R s ): (R d ,0,0,R s ) eTZ}. 

As a corollary of Theorem [2 we also have the following. 

Corollary 9: Let 1Z* ds be a closed convex set consisting of 
those rate pair (Rd, R s ) for which there exist auxiliary random 
variables (U, V) such that U o- V o- X o- (Y, Z) and 

R s < I(V;Y\U)-I(V;Z\U), (16) 
Rd > I(X;Z\U). (17) 

Then we have IZds = TZ*ds- 

Remark 10: The auxiliary random variable U in Corollary 
[9] only plays a role of time-sharing. Thus, the range of U 
may be assumed to satisfy \U\ < 2. The same remark is also 
applied for Corollary Q~2] 

Let 

C s = sup{R s : 3R d >0 s.t. (R d , R s ) e K ds } 

be the secrecy capacity, which can be characterized by the 
supremum of the rate R s for which there exists auxiliary 
random variable V such that V O X O (Y, Z) and 

R S <I(V:Y)-I(V-Z). (18) 

To achieve the rate given by the right hand side of Eq. ( fT8l . 
we conventionally used the following coding scheme. First, 
we construct a wire-tap channel code for channel pairs Py\v 
and Pz\v- Then, the code word in V™ is transmitted over 
prefixing channel P^\ v - If we simulate channel -Pjqy by using 
the channel simulation method J4), then we need randomness 
with rate H(X\vf[ By using this argument, we can derive 
the following inner bound on lZ ds that can be achieved 
by combining the ordinary wire-tap channel coding and the 
channel prefixing by the channel simulation method. 

Proposition 11: Let lZ* sim be a closed convex set consisting 
of those rate pair (R d ,R s ) for which there exist auxiliary 
random variables (U, V) such that U O V «-> X -s-> (Y, Z) 
and 



R s 
Rd 



< I(V;Y\U) 
> I(V;Z\U)- 



I(V;Z\U), 
H(X\V). 



(19) 
(20) 



Then we have lZ* sim C lZ ds . 

Since I(X; Z\U) = I(V; Z\U) + I(X; Z\V) < I(V; Z\U) + 
H(X\V) in general, the region lZ* ds is strictly broader than 
the region lZ* sim , i.e., the straightforward combination of the 
ordinary wire-tap channel coding and the channel prefixing by 
the channel simulation is suboptimal. 

Corollary 12: Suppose that the channel Py\x lS more capa- 
ble than Pz\x- Then the region lZ ds = W ds is a closed convex 
set consisting of those rate pair (R d , R s ) for which there exists 
an auxiliary random variable U such that U O X O (Y, Z) 
and 

Rs < I(X;Y\U)-I(X;Z\U), 
R d > I(X-Z\U). 

4 We are implicitly assuming that the empirical distributions of almost every 
code words are close to Py, which is true if we use the random coding 
method. 
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Fig. 2. The achievable region lids when Py\x i s BSC(O.l) and Pz\x is 
BSC(0.2). 

Moreover, it may be assumed that the ranges of U may be 
assumed to satisfy \U\ < 2. 

Proof: See Appendix [E] ■ 
As we can find from Corollary [12] we do not need auxiliary 
random variable V when the channel Py\x lS more capable 
than Pz\x- Thus, two regions 1Z* ds and TZ* im coincide. 

C. Numerical Examples 

First, we consider an example such that 1Z* ds and lZ* sim 
coincide. Suppose that Py\x and Pz\x are binary symmetric 
channels with crossover probabilities e\ and £2 respectively, 
where e\ < £2. In this case, Pz\x is degraded version of 
Py\x> which also implies that Py\x is more capable than 
Pz\x- Thus, we can apply Corollary [T2] Since the auxiliary 
random variable U only plays a role of time sharing, region 
IZds is the convex hull of the rates (Rd, R s ) satisfying 

R s < [h(p * £1) - h(ei)] - [h(p * e 2 ) - h(e 2 )], 
Rd > h{p * £ 2 ) - h(e 2 ) 

for some input distribution < Px(0) = p<l, where h(-) is 
the binary entropy function |f| and x* y = x(l — y) + (1 — x)y 
is the binary convolution. In Fig. [2] for the case with £1 = 0.1 
and £ 2 = 0.2 respectively, the region TZ,is is plotted. The input 
distribution achieving C s is the uniform distribution, and thus 
R s is constant when Rd > log 2 — h(0.2). By using a biased 
input distribution, R s can be positive even if Rd is smaller 
than log 2 - h(0.2). 

Next, we consider an example such that 1Z* ds and TV sim do 
not coincide. Suppose that Py\x is a binary symmetric channel 
with crossover probability e and Pz\x is a binary erasure 
channel with erasure probability S. When 4e(1 — £)log2 < 
5 log 2 < h(e), it is known that Py\x is not more capable than 
Pz\x 123 and C s > (fl For this example, we can compute 
the regions IZds = R-*ds as follows- Since TZ* ds is a convex set, 

5 Note that the base of the logarithm is e. 

6 Actually, for 4e(l — e)log2 < 5 log 2 < h(e), Pz\x m more capable 
than Py\x but Pz\x l& n °t l ess noisy than Py\x 1221 . Thus, I(X;Y) < 
I(X; Z) for every P x but there exists V such that I(V; Y) > I(V; Z), 
which means that C s > and V is needed to achieve C s - 
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Fig. 3. The achievable region lids = (solid line) and suboptimal 

inner bound TV" aim (dashed line) when Py\x ls BSC(O.ll) and Pz\x i s 
BEC(0.45). ^ 

for each R d , we can calculate max{R s : (Rd, R s ) G by 
minimizing 

max [I(V; Y\U) - I(V; Z\U) - n{I{X; Z\U) - R d )] (21) 

Puvx 

with respect to p, > 0, where p is the slope of the supporting 
line of TZ* ds . Since U only plays the role of the times sharing in 
Eq. (I2TI) . we can take U to be constant. Furthermore, by using 
the support lemma [6j, we can assume that |V| < \X\ = 2. 
Thus, Eq. (1211 can be calculated by exhaustive search of three 
parameters Py(0), P x \v(0\0), and P x \ v (l\l). Since P y (0) = 
i is not necessarily optimaO for Rd < (1 — 6) log 2, further 
reduction of parameters seems difficult. The region lZl im can 
be computed in a similar manner. 

In Fig. |2 for the case with e = 0.11 and S = 0.45 
respectively, the region lZ ds = lZ* ds and 7Z* sim are plotted. 



III. Channel Resolvability by Superposition 

In this section, we consider the channel resolvability prob- 
lem. The result in this section will be utilized in the direct 
part of the proof of Theorem [2] 

In the channel resolvability problem, we want to simulate 
the response PjJ by using channel P%\ x and as small number 
of uniform randomness as possible, where PJJ is the nth 
product of 

P z (z) = J2 p x(x)Pz\x(z\x) 

x 

for input distribution Px- The simulation is conducted by 
a deterministic map ip n : B n — > X n and uniform random 
number B n on B n . Let 

P Z ~(z n )= E Wl P S\x(^n(K)) 
b n £B n 1 " 

be the output distribution with map <p n . In this paper, the 
accuracy of the simulation is evaluated by the divergence 

7 When there is no constraint on Rd, it is known that fV(0) = 5 an d 
Px\v(0\0) = Px|v(l|l) are optimal (23). 
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criterion D(P~ Zn ||P£). It is well known WW\ that if 

R>I(X;Z), (22) 
then there exists a sequence of maps {tp n } satisfying 

Urn D{P 2n \\P2) = 0, (23) 



1 



lim sup — log \B„ | < R. 



(24) 



Typically, a sequence of maps realizing Eqs. d23l and 
( [241 is constructed by randomly generating \B n \ codeword 
x\ , . . . , x? B | according to P£ ■ We denote the generated code 
C n . Then we have the following proposition. 

Proposition 13: ( ifTTl ) For every n > 1, we have 



3 nV(e|-Pz|x,-Px) 



for < 8 < 1, where Ec n [•] means taking the average over the 
randomly generated code C n , and the function ip{0\Pz\x> Px) 
is defined by 

fK9\Pz\x,Px) 

In this paper we construct a sequence of maps realizing 
Eq. d23l by a different method. Let Pyx be a distribution 
such that the marginal is Px- We first randomly generate 
|A^2.n| codeword w", . . . , i>u^ 2 | according to the distribution 
Py. We denote the generated code by Ci, n - Then for each 
1 < i < \A42,n\, we randomly generate |.Mi )Tl | codeword 
xf 1: . . . , x^ Mi | according to the distribution Px\v('\ v i)- ^ e 
denote the generated code by C\. n . The empirical distribution 
of the codeword is given by 

P vnjtn (v n ,x n ) 



£ £ 

i€M2, n j£Mi,„ 



\M 2 , n \\Mi, n \ 



Py»K) 
P in]vn (x n \v n ) 



£ 



Pi 



Pyn(w") 

and the output distribution is given by 

P^(*") = E P v ^ n ,x n )P^ x {z n \x n ). 

For this construction, we have the following lemma. 
Lemma 14: For every n > 1, we have 



1 



< 



0|Ml,n| 



e'\M 2 , n \ 



*l>{e\p Z \x,Px\v,Pv) 



nip{6'\P z]v ,Pv) 



(26) 



8 Actually, slightly weaker statement, i.e., D{P^ n \\P^) in Eq. d23t is 
replaced by iD(P^„||P^), was proved in 1121 . The present statement can 
be derived from the result in II 11 . 



for < 6, 6' < 1, where E^ n £ 2 n [■] means taking the average 
over the randomly generated codes C\ n and 0%^, the function 

i>{Q\Pz\XiPx\ViPv) is defined as 



mPz\x,Px\v,Pv) 



= log]TP y («) 

^Px| V (xb)P Z |x(^W 1+9 Jp Z |y(^b)- e , 

(27) 

and ^(#|-, •) is defined in Eq. (t25T >. 

Proof: See Appendix lAl ■ 

Corollary 15: If i? x > I(X;Z\V) and i? 2 > I(V;Z), 
there exists a sequence of map tp n : M.\. n x M.2,n — > X n 
such that 



lim D(P 2n \\PS) = 0, 

n— too 

lim sup — log |A^i 5n | < Ri, 
n—>oc Tl 

limsup - log \M2,n\ < R2- 
n— >oo Yl 



(28) 
(29) 
(30) 



Proof: See Appendix iBl ■ 
From Corollary [15] we find that the channel resolvability 
coding scheme proposed in this section can achieve the rate 
shown in Eq. <G3, i.e., I(X;Z) = I(V;Z) + I(X;Z\V). 
Splitting the randomness into two part does not have any 
meaning in the channel resolvability coding, but as we will 
find in Section HV-AI this coding scheme does have meaning 
when we send the confidential message. 

IV. Proofs of Main Results 
A. Proof of Direct Part of Theorem [2] 

We prove the direct part of Theorem [2] by using the result 
in Section [HI] The direct part of the theorem follows from the 
following Lemma [TBI and Lemma [T71 

Lemma 16: Let iz( m *> be a closed convex set consisting 
of those quadruples (Rd, Po, Pi, R s ) f° r which there exist 
auxiliary random variables (U, V) such that (/Hl/f>If> 
(Y, Z) and 



Po 


< 


I(U;Z), 


(31) 


i?l + Rg 


< 


I(V;Y\U), 


(32) 


Ro + i?i + R s 


< 


I(V;Y), 


(33) 


Ri 


> 


I(V;Z\U), 


(34) 


Ri! 


> 


I(X;Z\V). 


(35) 



Then 1Z ( "^ C K. 

We note the following observation. From the definition of 
the problem, if 



(Rd ~ r d , Ro + r , Pi - r 



r d ,R s +r s ) e TZ 



for some r d , ro, r s > 0, then we also have (Rd, Ro, Pi, P s ) £ 
1Z. By using this argument, we have the following. 
Lemma 17: We have 1Z* C K. 

Proof: See Appendix ICl ■ 
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We now prove Lemma [T6] For a while, we consider the 
case with n = 1 and omit the superscript and subscript to 
simplify the notation. For each common message k G JC, we 
randomly generate codeword Uk according to distribution Pjj. 
We denote such a code Co- For each k and for each (£, s) G 
C x S, we randomly generate codeword v^is according to 
distribution Pv\u('\ u k)- We denote such a code Ci- For each 
(k, £, s) and for each a G A we randomly generate codeword 
ccfc^ a according to distribution (•|^fe£s)- We denote such 
a code Ci. 

For real numbers ao,ai,ct2 > specified later, let 

To = {(u,z):P zlu (z\u)>e a «P z (z)}, 

Ti = {(u,v,y):P Ylv (y\v)>e Q ip Ylu (y\u)}, 

T 2 = {(v,y) :P Y \ v (y\v) >e a *P Y (y)}, 

and let T = 7i fl X 7i)- Eve's decoding region is defined 
by 

P fc = {z : (u fc , z) G 7o, (Ujj, z) £%V~k^ k}, 

i.e., 0(z) = fc if z G Bob only decode (fc, £, s) and he 
does not decode dummy randomness a G A. Bob's decoding 
region is defined by 

T>kis = {y ■ (uk,vkis,y) g T, 

(Vkte'V) iTV(k,£,s) ± (k,£,s)}, 

ie., = (k,£,s) if y G XW 

By the above code construction, we have the following. 
Lemma 18: We have 

Ec CiC 2 [-Perr (/,#)] 

< Pc/VY(7; c )+Pyr(7?) 

+ |£||S|e- Ql +|/C||/:||5|e- Q2 , (36) 

E CoCl c 2 [-Per, UM 

< Puz(To c ) + \]C\e' a " (37) 



and 



E CoCl c 2 [D(f)} < 



0\A\> 



M8'\Pz\v,Pv\u,Pv) 



(38) 



for < 6,6' < 1, where the functions ip(6\-,-,-) is defined by 
Eq. (EJ. 

Proof: See Appendix iDl ■ 
We apply Lemma [18] for asymptotic case. For 
{R d ,Ro,Ri,Rd) G /^. {m) and arbitrary small 6 > 0, 
we set |/C„| = [e™ (% - 25) J, |£„| = Le" (fll+25) J, 
|5„| = [e"(^- 45 )j, \A n \ = [ e "(^+ 25 )j, ao = I(U;Z) - S, 
ai = I(V; Y\U) -6, oi2 = I(V; Y) - 5. Then, 

\C n \\S n \e^ n < e -nWV;Y\U)-Ri-R.+S) } 
\lC n \\C n \\S n \e- a * n < e -n(/(y ; r)-i? -fl 1 -iJ s +35) ; 

\K„\e~ a ° n < e -n(I(U;Z)-Ro+S) 

'if z f T>k for every k £ /C, we set </>(z) = 1, which is not important in 
our analysis of error probability. A similar remark is also applied for Bob's 
decoder. 



converge to asymptotically. Furthermore, by the law of 
large numbers, P£ vy (7?;J, P$ Y (T 2 C J, and P$ Z (T %) also 
converge to asymptotically. 

Since tp'(0\P z \x, Px\v, p v) = I(X;Z\V), there exists 
6q > such that 



il>(Oo\Pz\x,Px\v,Pv) 



which implies 



<I(X;Z\V) + S <R d + S, 



Thtis. 



--\og\A n \ + i:(6o\P zl x,Px\v,Pv) < S. 
n 



o ni,(9 \P z]x ,P xlv ,P v ) 



6 \A n \ e ° 



exponentially converges to asymptotically. Similarly, since 
ip'{0\P z \v,Pv\u,Pu) = I(V;Z\U), there exists 6' Q > such 
that 



m\p^uM <mm+s<Ri+Si 

which implies 

-^■]og\C n \+i>(0' Q \P x \ v ,P v \ u ,Pu) < -6. 



Thus, 



s nil>(S' \P x]v ,P v]u ,Pu) 



exponentially converges asymptotically. This completes the 
proof of Lemma [16] ■ 



B. Proof of Converse Part of Theorem [2] 

Suppose that (Rd, Ro, Ri, R s ) G 1Z. Then, for arbitrary 7 > 
0, there exists n such that 

n(Ro--f) < log|/C„|, 

n(R + Ri + R s - 7) < log|X:„||£„||5 n |, 

n{R s --y) < log|5n|, 

n{R 1 +R d + 1 ) > log|A.||A|, 

n{R d + 1 ) > log I Ail- 

By combining these inequalities with the following Lemma 
[19] and Lemma [20] we have the converse part of the theorem. 
The statement about the range sizes of U and V can be proved 
exactly in the same manner as J6] Theorem 17.13]. It should 
be noted that Eqs. (t9b — (TTTb are derived in the same manner 
as J6] Theorem 17.13] and the construction of the auxiliary 
random variables are also the same. Eqs. (TT2T) and ( TT3T ) are 
additionally proved in this paper by using the fact that the 
encoder is deterministic given the dummy randomness. 
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Lemma 19: There exists e n — > such that 

log \JC n \ 

< I(K n ',Y n )+ne n , 

log \K n \ 

< I(K n ;Z n )+ne n , 
log |/C n ||£ n ||5 n | 

< I(K n ,L n ,S n ;Y n )+ne n , 
log |/C„||£„||<S„| 

< I(L n ,S n ; Y n \K n ) + I(K n - Z n ) + 2ne n , 
log|5„| 

< I(L n , S n ; Y n \K n ) - I(L n , S n ; Z n \K n ) + 4ne„, 
log |£„||A»| 

> I(X n ;Z n \K„)-2ne n , 
log | AJ 

> I(X n ;Z n \K n ,L n ,S n ). 

Proof: By using Fano's inequality, we have 

log|/C„| = H{K n ) 

= I(K n ; Y n ) + H(K n \Y n ) 
< I(K n ;Y n )+ne n , 

and 

log|/C„| < I(K n ;Z n )+ne n . 
By using Fano's inequality, we also have 
log |/C„||£„||<S n | 

< I(K n ,L n ,S n ;Y n )+ne n 

and 

log|/C„||£„||S„ 

= H(L n ,S n \K n )+H(K n ) 

= I(L n , S n ; Y n \K n ) + I(K n ; Z n ) + 2ne n . 

By using the security condition and Fano's inequality, we have 

I(S n ;Z n \K n ) 

= I(S n ,K n ;Z n )-I(K n ;Z n ) 

= I(S n ; Z n ) + I(K n ; Z n \S n ) - I(K n ; Z n ) 

< I{S n -Z n )+H(K n \Z n ) 

< 2ne n . (39) 
By using Fano's inequality and by using Eq. ( 139) . we have 

log \S n \ 

= H{S n \K n ) 

< I(S n ;Y n \K n )+ne n 

= I(Ln, S n ; Y n \K n ) — I(L n ; Y n \S n , K n ) + ne n 

< I{L n: S n ; Y n \K n ) - H(L n \S n ,K n ) + 2ne n 

< I(L n , S n ; Y n \K n ) - I(S n ; Z n \K n ) 
-H(L n \S n ,K n ) + 4ne n 

< I(L n , S n ; Y n \K n ) - I(L n , S n ; Z n \K n ) + Ane n . 



By noting that f n is a deterministic function and by using 
Eq. (1391 . we have 

log|£ n ||Ai| 

> H(X n \K n ,S n ) 

> I(X n ;Z n \K n ,S n ) 

= I(X n , S n ; Z n \K n ) - I(S n ; Z n \K„) 

> I{X n ;Z n \K n )-2ne n . 

Finally, by noting that /„ is a deterministic function, we have 

log | A, | > H(X n \K n ,L n ,S n ) 

> I(X n ; Z n \K n , L n , S n ). 



Lemma 20: For fixed n, let T be the random variable that is 
uniformly distributed over {1, . . . , n} and independent of the 
other random variables. Define the following auxiliary random 
variables 

U t = (K n ,Yt\Z? +1 ), 
V t = (L n ,S n ,U t ), 
U = (U T ,T), 
V = (V T ,T). 

Then, we have 



±I{U;Y)], 
I(K n ;Z n ) 



and 



I(K n -Y n ) 

< nI(U;Y), 
I(K n ;Z n ) 

< nI(U;Z), 

I{Krn Lni S n ', Y n ) 

< n[I(V;Y\U) 
I(L„,S n ;Y n \K„) + 

< n[I(V;Y\U) 
I(L„, S n ', Y n \K„) — I(L n , S„', Z n \K n ) 

< n[I(V;Y\U)-I(V;Z\U)], 
I(X n ;Z n \K n ) 

> nI{X;Z\U), 
I(X n ; Z n \K n , L n , S n ) 

> nI(X;Z\V) 



U o X «-> (Y,Z). 



(40) 
(41) 
(42) 
(43) 
(44) 
(45) 
(46) 

(47) 



Proof: Since the proof of Eq. d47l i is well known J6], we 
only prove the other inequalities. 
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a) Proof of Eq. SZUb : 
I{K n -Y n ) 

n 

= "£l(K n ;Y t \Y^) 



t=i 



< Y,I(K n ,Y*-\Z? +1 ;Y t ) 



t=i 



t=i 

= nI(U T ;Y T \T) 
= nI(U T ,T;Y T ) 
= nI(U;Y). 

b) Proof of Eq. g7]): 

I(K n ;Z n ) 

n 

= £i(ir n ;z t |z? +1 ) 

n 

t=i 

n 

= £/(E/ t ;Z t ) 

t=i 

= nI{U T -Z T \T) 

= nI(U T ,T;Z T ) 
= nI{U;Z). 

c) Proof of Eq. $42}: 

I(K n , L n , S n ; Y n ) 



= ^2 1{K n ,L n , S n ; YtlY^ 1 ) 
t=i 

n 

< '5~] I{K n , L n , S n , y% 1 , Z™ +1 ;Y t ) 
t=i 

n 

t=i 

= n/(C/ T ,^T;F T |T) 

= nI(U,V;Y) 

= n[I{V-Y\U) + I(U-Y)]. 



d) Proof of Eq. 

I(L n ,S n ;Y n \K n ) + I(K n ;Z n ) 

n 

= 5^[/(L ni 5„; F t |X„, F/" 1 ) + Z t |Z t " +i ; 

t=l 

77 
i=l 

+I(Z™ +1 ; Y t \K n , Yt 1 ) + Z T |Z t n +1 )] 
t=l 

+/(r i t - 1 ; Z t |A'„, + J(if n ; Z t |Z t » +1 )] 



< /liHLn, Sn]Yt\K n ,Y* 1 ,Z™ +1 ) 
t=l 

+I(K n ,Yt\Z? +1 ;Z t )} 

n 

= Y^[I(Y t ;Y t \U t )+I(U t ;Z t )] 
t=i 

= n[I(V T ;Y T \UT,T) + I(U T ;Z T \T)} 
= n[I(V;Y\U) + I(U;Z)}, 



where we used Csiszar's sum identity |24| in (a). 
e) Proof of Eq. $44$: 

I(L n , S n ;Y n \K n ) - I(L n , S n ; Z n \K n ) 

n 

= ^""^ [I(L n , S n ; Yt | K n , Y* 1 ) — I(L n , S n ', Z t \K n , Z™ +1 )] 



Y)l{L n ,S n \Y t \K n X~ X ) 



~\~I{Zt+i> Yt\K ni L n , S n , 
I{L n , S n \ Z t \K n ,Z? +1 )} 

n 

t=i 

~I{L n ,S ni Y 1 ; Zt\K n , Z™ +1 )] 

n 

= ^"^(-^n; <9ni Yt\K n , Yj 1 ,Z™ +1 ) 
t=l 

+I(Z? +1 ;Y t \K n , Yt 1 ) - /(F/- 1 ; Z t |Jf n , 

I{L n , S n ; Z t \K n ,Yt\Z? +1 )} 
n 



—I(L n , S n ] Zt\K 7ll , ^t+i)] 
n 

= ^[/(^FlC/i)-/^;^!^)] 
t=i 

= n[7(y T ; F T |C/ t , T) - /(Ft ; Z T | E/t , T)] 
= n[/(l/;F|C/)-/(^;Z|C/)], 

where (a) and (b) follow from Csiszar's sum identity l24l . 
f) Proof of Eq. $45}: 

I(X n ;Z n \K n ) 

n 

= J2l H ( Z t\Kn, Z? +1 ) - H(Z t \K n ,X n , Z\\ x )\ 

t=l 

(a) n 

> Yy H ( Z t\ K n,Yt\Z? +r )-H{Z t \Kn,X u Yt\Z? +x ) 

t=l 

n 

t=l 
7 1 

= ^2l(X t ;Z t \Ut) 
t=i 

= nI(X T ;Z T \U T ,T) 
= nI(X;Z\U), 
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where (a) follows from the fact that 

rn 
L t+1 



(K n ,Xl- 1 ,X? +1 ,Y[- 1 ,Zt+i), X t , and Z t form Markov 



chain. 



g) Proof of Eq. 

I(X n ; Z n \K n , L n , S n ) 

n 

= yjH(Z t \K n ,L n , S n , Zt +1 ) 
t=i 

—H(Z t \K n , L n , S n , X n , Z" +1 )] 

(a) n 

> J2l H ( Z t\ K n,L n ,S n ,Yt\Z? +1 ) 



t=l 



—H(Zt\K n , L n , S n , Xt,Y* 1 , Z™ +1 )] 

n 

= ^~]l(Xt] Zt\K n , L n , S n ,Y* 1 ,Z™ +1 ) 

n 

= Y,I{X t ;Z t \V t ) 
t=l 

= nI{X T ;Z T \V T ,T) 
= nI(X;Z\V), 



where (a) follows from the fact that (K n , L n , S n , X[ \ Xf +1 , 
Y*' 1 , Z? +1 ), X t , and Z t form Markov chain. ■ 

V. Conclusion 

In this paper, we investigated the trade-off between the rate 
of the random number, the rates of common, private, and 
confidential messages. 

As a by-product of our result, Lemma[T4lcan be also applied 
to the three receiver wire-tap channel, and the lower bound 
of secrecy capacity obtained in JT] Corollary 1] with strong 
security can be proved. 
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Appendix 

A. Proof of Lemma [74] 

For simplicity of notation, we only prove the statement for 
n = 1, and the subscript n is omitted in the proof. The 
statement for n > 2 can be proved by regarding the nth 
product distribution as one distribution and by noting that 

1>(0\PS\x,Px\v> p v) = ™/>{0\Pz\x,Px\v,Pr) 



and 



hold. 



W\Pz\viPv) = nW\Pz\v,Pv) 



We first note the following observations. By taking average 
over the randomly generated codes C\ and C2, we have 



Py(v)P m ( X \v) 

E Cl c 2 
Pvx(v,x) 



E E \ M ^\M2\ 1[Vi = v,Xii = x] 



and 



Ec lC2 [Py(«)] = Ec 2 [Pv(v)] 



E ja^i 1 ^"] 



Pv{v). 



(48) 



Furthermore, for fixed C2, by taking the average over the 
randomly generated code Ci, we have 



p x\v( x \ v ) 

J2i£M 2 YjjeMi \Mi\\M 2 \ = V,: 



Er 



Py(v) 



P V(v) 

= Px\v(x\v) (49) 

Let Pz' be the output distribution when the input distribu- 
tion is Py(v)Px\v( x \ v )- Then, from Eq. (|49l , we have 

E Cl [P s (z)]=P z ,(z) 

for every z£Z. Thus, we have 

E Cl c 2 [D{P- Z \\P Z )\ 

P-zW 



= E, 



CiC 2 



E p ^( z ) lo g 



Pz(z) 



E r 



+ E Cl 



Z V ' 



-C1C2 



J2 p z( z )^g 



P Z (z) 



Ec 2 



p z\z) 

Pz>{z) 
p z{z) 



= E Cl e 2 [D(Pz\\Pz')}+Kc 2 [D( p z'\\Pz)}. (50) 



We bound each term of Eq. d50t by using Proposition \13] 
By the monotonicity of the divergence, we have 

D(P Z \\P Z ') < D(P IZ \\P IZ ') 

= E m^V-wii^voi^)), 

where 



JOURNAL OF MTeX CLASS FILES, VOL. 6, NO. 1, JANUARY 2007 



11 



and 



PiZ'(i,z) = TTTl P z\v(z\vi) 



Thus, by taking average over C\ and by using Proposition [13] 
for input distribution Px\v('\ v i) instead of Px, we have 



E Cl [D(P 2 \\P Z >)} 



< 



< 



E 

i£M 2 



1 



y -1 1 



iEM 2 



DiP^miPzwi-h)) 

e 4>{6\Pz\x,Px\v{-\v<,))_ 



By taking average over C2 and by noting Eq. (1481 1. we have 

E ClC2 [D{P 2 \\P Z .)\ ^ 

_ e 4>(8\P z \x,Px\v(-\v)) 

■ " ' d\Mi\ e 

1 



\Mi\ 



-Mil 

■4>(6\Pz\x,Px\v,Pv) 



On the other hand, by using Proposition Qj] for input 
distribution Py and channel Pz\v> we have 



1>(6'\P Z]V ,P V ) 



E C2 [DiP^P^K^j^e 



B. Proof of Corollary [75] 

We can choose 7 > such that Ri > I(X; Z\V) + 2-/ and 
R2 > I(V;Z) + 2j. Let \Mi, n \ = [e nRl \, \M 2 ,n\ = [e nR ' 2 \. 
Since tp'(0\P z \x, Px\v , Pv) = I(X;Z\V), there exists 9 > 
such that 

1>(Po\Pz\x,Px\v,Pv) 



< I(X;Z\V) + 7 < 



which implies 



■log|Ml,n|+^0|i > Z|Jt,Px|V.JV) < -7<0. 



Thus, the first term of Eq. (I26l i converges to asymptotically. 
Similarly, we can show that the second term of Eq. (I26t 
converges to asymptotically. Thus, we have the assertion 
of the corollary. ■ 

C. Proof of Lemma \17\ 

Although the lemma can be systematically proved by using 
the Fourier-Motzkin elimination, we explicitly find (rd, ro,r s ) 
satisfying 

{R d - r d , Ro + r ,Ri - r - r s + r d , R s + r.) G TZ (m) (51) 

for given (Rd, Ro, Ri, R s ) S TV as follows. 

If Ri + R s < I(V; Y\U) and R 1 < I(V; Z\U), we set 

r d :=I(V;Z\U)-R 1 

and (ro, r s ) := (0, 0). Then, Eqs. (|3H . and (l34l i are obviously 



satisfied. Eq. (I32t can be confirmed as 

R 1 +r d + R s < I(V;Z\U)+I(V;Y\U)-I(V;Z\U) 
= I(V;Y\U), 



Eq d3~3l can be confirmed as 

i?o + J2i + r d + R s < I(U;Y)+I(V-Y\U) 
= I(V;Y), 

and Eq. d35l l can be confirmed as 

Rd-r d = R d + Ri-I(V;Z\U) 
> I(X;Z\U) - I(V;Z\U) 
= I(X;Z\V). 

Thus, Eq. (EB holds. 

If Ri+ R s < I(V;Y\U) and R ± > I(V;Z\U), we set 
(rd, r i, r s) '= (0,0,0). Then, Eq. ( BTT l obviously holds. 
If Ri + R s > I(V; Y\U), we set r d := and 

r s := I(V;Y\U)-I(V;Z\U)-R 8 , 

r := i? 1 + i? s -/(y ; r|[/) 

= (R 1 -r s ) + (R s + r s )-I(V;Y\U) 

= Rj.-r.-IiViZlU). (52) 



Then, Eqs. (l32l . (l33l l, and (l35l l are obviously satisfied. Eq. (l3TT l 
can be confirmed as 



flo + ro = i2o + i2i + i2.-/(V;y|EO 
< /(E/;Z), 

and Eq. d34l i can be confirmed from Eq. (l52l . Thus, Eq. (IBTt 
is satisfied. 



Z). Proof of Lemma [JS] 



aj Proof of Eq. \3m : We first note the following obser- 
vations. By taking the average over randomly generated codes, 
we have 



E Co CiC 2 [Perr(f,g)] 



-C0C1C2 



-C0C2 



E 



C0C2 



k,£„ 



E 

k.l.s.o 



\K\\C\\S\\A 



PY\x(Pkls\ x ktsa, 



\K\\C\\S\\A 



■E Cl [Py| X (^ s |^ sa )] 



(53) 
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Let Tuv = {y ■ (u,v,y) G T}. Then, we have 



Note that the relation 



E, 



CoC 2 



kJ. 



< E, 



•CaCi 



E \)C\\£\\S\ { PY \ V (Tu k v kia \ v Ms) 



k,e. 



S P Y\v{Tu k v k Jv k ls)} 
{k,l,s)=£(k,l,s) 



Y P Y\v{T Uk v kli \vkls) 



k^-k t,a 



\s\ 

+D(P 2lK (-\k)\\P z]u (-\u k )) 

= T,^ D ( P Z\K S (-\ k ^)\\ P Z\u(-\Uk)) 

holds for each k € JC. Thus, by using Lemma [14] for 
p v\u('\ u k) instead of Py, we have 



Ec lC2 [D(f)j 

< 



y- 



e\A\' 



a i'(0\P z \x,Px\v ,Pv\u{-\u k )) 



1 



0'|£| £ 



M8'\Pz\v,Pv\u(-\u-k)) 



By taking average over Cq and by noting 



E, 



^ EjjqT^'W'P) 

+|£||5| p uv(u, v)P YW {T w \u) 
+\K\\C\\S\Y, p uv(u,v)P Y {T uv )} 

< Puvy{T c ) + \£\\S\e.- ai + \IC\\C\\S\e' a \ 
where we used 

P Y \u(y\u) < P Y \v(y\v)e- a \ 
Py{y) < P Y \v(y\v)e- a i 

for y G Tuv in the last inequality. 

b) Proof of Eq. 071 ) : In a similar manner as Eq. ( l53t . 
we have 



E|^|i[«* = «] 



we have 

E CoClC2 [£>(/)] 
< J2 p u(u) 

U 

1 



0|.4|' 



^(fll-P^lx.-Pxiv.-FVlt/M")) 



- i>(e'\p Z f V ,p vlu (-\u)) 



1 



^{e\Pz\x,Px\v-Pv) 



i>(8'\P z \v,Pv\u,Pu) 



Ec oCl C 2 [Perr(/,0)] =Ec„ 



which is just a random coding error probability of channel 
Pz\u- Thus, by the standard arguments of the information 
spectrum approach 1251 , we have Eq. ( |37l >. 

c) Proof of Eq. I[38l : By the monotonicity of the diver- 
gence, we have 

D(P S z\\P s x P- z ) 

< D(P KS2 \\P S x P K2 ) 

= T.^ p sz\Kim\ p s^ p Z \ K {-\k)) 



k , s 



+ 6'\O e ' e 



E. Proof of Corollary [72] 

By noting that U, V, X, and (Y, Z) form Markov chain, 
which implies I(V; Y\X, U) = and I(V; Z\X, U) = 0, we 
have 

I[V;Y\U)-I(V;Z\U) 

= I(V,X;Y\U)-I(V,X;Z\U) 

-[I(X;Y\U,V)-I(X-Z\U,V)} 
= I(X;Y\U)-I(X;Z\U) 

-[I(X;Y\U,V) - I(X;Z\U,V)}. 

Since Py\x is more capable than Pz\x< we nave 
I{X; Y\U = u, V = v) - I(X; Z\U = u,V = v) > 

for every (u, v), which implies 
I(V; Y\U) - I(V; Z\U) < I(X; Y\U) - I(X; Z\U). 

Thus, the auxiliary random variable V is not needed. ■ 
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